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WHICHEVER IS LONGER, FROM THE MAILING DATE OF THIS COMMUNICATION. 
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earned patent term adjustment. See 37 CFR 1 .704(b). 
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3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 
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4) ^ Claim(s) 1-21 is/are pending in the application. 
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DETAILED ACTION 



Response to Amendment 

1 . This action is responsive to applicant's amendment and remarks received on 1 1/8/07. 
Claims 1-21 are currently pending. 

Claim Objections 

2. In response to applicant's amendment of claims 14, 15, 16, 17, and 18, the previous claim 
objections are withdrawn. 

Claim Objections - 37 CFR L 75(a) 

3. In response to applicant's amendment of claim 13, the previous claim objection is 
withdrawn. 

Claim Rejections - 35 USC § 112 

4. The following is a quotation of the second paragraph of 35 U.S.C. 1 12: 

The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the 
subject matter which the applicant regards as his invention. 

5. Claim 20 is rejected under 35 U.S.C. 1 12, second paragraph, as being indefinite for 
failing to particularly point out and distinctly claim the subject matter which applicant regards as 
the invention. 
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Regards to claim 20, the phrase, "salient part of an image", is interpreted broadly as 
being a "region of interest of an image". What is the scope of the two phrases? "Salient part" 
suggests a prominent/striking part of an image. A prominent/striking part of an image can be a 
face, a particular background/object, etc. The salient part deems the claim to be vague and 
indefinite. For examination purposes, the broadest interpretation will be considered taken into 
account. Further clarification and correction is required. 

Claim Rejections - 35 USC § 102 

6. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the 
basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(a) the invention was known or used by others in this country, or patented or described in a printed publication in this 
or a foreign country, before the invention thereof by the applicant for a patent. 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public use or on 
sale in this country, more than one year prior to the date of application for patent in the United States. 

(e) the invention was described in (1) an application for patent, published under section 122(b), by another filed 
in the United States before the invention by the applicant for patent or (2) a patent granted on an application for 
patent by another filed in the United States before the invention by the applicant for patent, except that an 
international application filed under the treaty defined in section 351(a) shall have the effects for purposes of this 
subsection of an application filed in the United States only if the international application designated the United 
States and was published under Article 21(2) of such treaty in the English language. 

7. Claims 1, 10, 12 are rejected under 35 U.S.C. 102(b) as being anticipated by Jun et al 
(US 2001/0020981 Al). 

Regarding claim 1, Jun teaches a method for generating a highly condensed visual 
summary of video regions, comprising: 

determining a dominant group in each of a plurality of video segments (Jun: paragraphs 
[0023]-[0024]); 
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determining a key frame in each of the video segments (Jun: paragraphs: [0023]-[0024]); 
defining a germ associated with each dominant group in each of the video segments (Jun: 
paragraph [0051]); 

laying out the germs on a canvas, each germ associated with a support; and filling in the 
space of the canvas (Jun: figures: 13a, 13b, 17). 

Regarding claim 10, Jun teaches a method for generating a highly condensed visual 
summary of video regions, comprising: 

determining a germ in each of a plurality of images, the germ containing a region of interest 
(Jun: paragraph [0051]); 

laying out the germs on a canvas, each germ associated with a support; and filling in the 
space of the canvas with one or more parts of the image from the support (Jun: figures: 13a, 13b, 
17). 

Regarding claim 12, Jun teaches receiving user input, the user input associated with a 
part of an image (Jun: paragraph [0077]). 

Claim Rejections - 35 USC §103 
8. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: • 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 
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9. Claims 2-6, 13-15, 20 are rejected under 35 U.S.C. 103(a) as being unpatentable over Jun 
et al (US 2001/0020981 Al) in view of Uchihashi (ACM Multimedia: "Video Manga: 
Generating Semantically Meaningful Video Summaries"). 

Regarding claim 2, Jun discloses all elements as mentioned above in claim 1. Jun does 
not teach determining a group within each of the plurality of video segments having the largest 
volume. 

Uchihashi teaches determining a group within each of the plurality of video segments 
having the largest 3-D volume (Uchihashi: section 4.2, length of the segment is scored). 

It would have been obvious at the time the invention was made to one of ordinary skill in 
the art to modify the Jun reference to determine a group having the largest volume as taught by 
Uchihashi, in order to "calculate an importance score for each segment based on its rarity and 
duration" since "a segment is deemed less important if it is short or very similar to other 
segments" (Uchihashi: section 4.2). 

Regarding claims 3, 4, and 20, Jun discloses all elements as mentioned above in claim 1 . 
Jun does not teach defining a two dimensional shape that encompasses the projection of the 
dominant group onto the key frame; wherein the two dimensional shape is a rectangle; and using 
an algorithm to determine a salient part of an image. 

Uchihashi teaches defining a two dimensional shape that encompasses the projection of 
the dominant group onto the key frame (Uchihashi: figure 2; section 4.4) and wherein the two 
dimensional shape is a rectangle (Uchihashi: figure 2; section 4.4). 

It would have been obvious at the time the invention was made to one of ordinary skill in 
the art to modify the Jun reference to define a two dimensional shape that is a rectangle as taught 
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by Uchihashi, in order to "form a pictorial abstract of the video sequence" where a "sequence of 
frames .... fills space efficiently and represents the original video sequence well" (Uchihashi: 
section 4.4). 

Uchihashi further teaches using an algorithm to determine a salient part of an image 
(Uchihashi: figure 4.2). 

It would have been obvious at the time the invention was made to one of ordinary skill in 
the art to modify the Jun with Uchihashi combination as mentioned above to determine a salient 
part of an image as taught by Uchihashi, "to select appropriate keyframes for a compact pictorial 
summary" (Uchihashi: section 4.2). 

Regarding claims 5 and 6, Jun with Uchihashi discloses all elements as mentioned above 
in claim 3. Jun with Uchihashi as mentioned in claim 3, does not teach determining a scale 
factor to be applied to every germ such that the germs are scaled to the maximum size that fits 
into the canvas and placing the germs in rows, wherein each row has a height according to the 
longest germ in the particular row. 

Uchihashi further teaches determining a scale factor to be applied to every germ such that 
the germs are scaled to the maximum size that fits into the canvas (Uchihashi: section 4.3, 4.4) 
and placing the germs in rows, wherein each row has a height according to the longest germ in 
the particular row (Uchihashi: figure 2), 

It would have been obvious at the time the invention was made to one of ordinary skill in 
the art to modify the Jun with Uchihashi combination to place the germs in a row as taught by 
Uchihashi, to "fill space efficiently and represent the original video sequence well" (Uchihashi: 
section 4.2). 



Application/Control Number: Page 7 

10/815,354 

Art Unit: 2624 

Regarding claim 13, Jun discloses all elements as mentioned above in claim 10. Jun does 
not disclose using an algorithm to determine the regions of interest of an image based on one or 
more methods selected from the group consisting of a general image analysis algorithm, a face- 
detection algorithm, and object detection algorithms and user input. 

Uchihashi teaches using an algorithm to determine the regions of interest of an image 
based on one or more methods selected from the group consisting of a general image analysis 
algorithm (see section 4.2, segment is scored and weighted), a face-detection algorithm, and 
object detection algorithms and user input. 

It would have been obvious at the time the invention was made to one of ordinary skill in 
the art to modify the Jun reference to determine the regions of interest with a general image 
analysis algorithm as taught by Uchihashi, "to select appropriate keyframes for a compact 
pictorial summary" (Uchihashi: section 4.2). 

Regarding claims 14 and 15, Jun reference discloses all elements as mentioned above in 
claim 10. Jun reference as mentioned in claim 10, does not teach determining a scale factor to be 
applied to every germ such that the germs are scaled to the maximum size that fits into the 
canvas and placing the germs in rows, wherein each row has a height according to the longest 
germ in the particular row. 

Uchihashi further teaches determining a scale factor to be applied to every germ such that 
the germs are scaled to the maximum size that fits into the canvas (Uchihashi: section 4.3, 4.4) 
and placing the germs in rows, wherein each row has a height according to the longest germ in 
the particular row (Uchihashi: figure 2). 
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It would have been obvious at the time the invention was made to one of ordinary skill in 
the art to modify the Jun reference to place the germs in a row as taught by Uchihashi, to "fill 
space efficiently and represent the original video sequence well" (Uchihashi: section 4.2). 
10. Claims 7-9, 16-18 are rejected under 35 U.S.C. 103(a) as being unpatentable over Jun et 
al (US 2001/0020981 Al) in view of Li et al (US 2006/0023786 Al). 

Regarding claims 7-9, Jun discloses all elements as mentioned above in claim 1 . Jun 
does not teach assigning a pixel value of each point in the canvas to the same pixel value in the 
support associated with the germ closest to each point; wherein if the germ closest to the point 
does not have a support that includes the point, the point is assigned the pixel value of the closest 
germ with a support that includes the point; wherein the point is assigned a background value if 
no support includes the point. 

Li teaches assigning a pixel value of each point in the canvas to the same pixel value in 
the support associated with the germ closest to each point (Li: paragraph [0144]); wherein if the 
germ closest to the point does not have a support that includes the point, the point is assigned the 
pixel value of the closest germ with a support that includes the point (Li: paragraph [0052]); 
wherein the point is assigned a background value if no support includes the point (Li: paragraph 
[0052], [0144]). 

It would have been obvious at the time the invention was made to one of ordinary skill in 
the art to modify the Jun reference to assign pixel value to a germ that is closest as taught by Li, 
to create an aesthetically/visually pleasing to the user by removing white spaces between germs. 

Regarding claims 16-18, Jun discloses all elements as mentioned above in claim 10. Jun 
does not teach assigning a pixel value of each point in the canvas to the same pixel value in the 
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support associated with the germ closest to each point; wherein if the germ closest to the point 
does not have a support that includes the point, the point is assigned the pixel value of the closest 
germ with a support that includes the point; wherein the point is assigned a background value if 
no support includes the point. 

Li teaches assigning a pixel value of each point in the canvas to the same pixel value in 
the support associated with the germ closest to each point (Li: paragraph [0144]); wherein if the . 
germ closest to the point does not have a support that includes the point, the point is assigned the 
pixel value of the closest germ with a support that includes the point (Li: paragraph [0052]); 
wherein the point is assigned a background value if no support includes the point (Li: paragraph 
[0052], [0144]). 

It would have been obvious at the time the invention was made to one of ordinary skill in 
the art to modify the Jun reference to assign pixel value to a germ that is closest as taught by Li, 
to create an aesthetically/visually pleasing to the user by removing white spaces between germs. 

1 1 . Claims 11, 19 is rejected under 35 U.S. C. 103(a) as being unpatentable over Jun et al 
(US 2001/0020981 Al) in view of Li et al (US 7,035,435 B2). 

Regarding claim 11, Jun discloses all elements as mentioned above in claim 10. Jun does 
not teach detecting a face in each of the plurality of images. 

Li teaches detecting a face in each of the plurality of images (Li: col. 7, lines 33-51). 

It would have been obvious at the time the invention was made to one of ordinary skill in 
the art to modify the Jun reference to detect a face as taught by Li, in order to determine the 
importance of a frame since "a human face will be more informative than, for example, a 
landscape frame" (Li: col. 7, lines 33-51). 
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Regarding claim 19, Jun discloses all elements as mentioned above in claim 1. Jun does 
not teach detecting a face in each of the plurality of images. 

Li teaches detecting a face in each of the plurality of images (Li: col. 7, lines 33-51). 

It would have been obvious at the time the invention was made to one of ordinary skill in 
the art to modify the Jun reference to detect a face as taught by Li, in order to determine the 
importance of a frame since "a human face will be more informative than, for example, a 
landscape frame" (Li: col. 7, lines 33-51). 

12. Claim 21 is rejected under 35 U.S.C. 103(a) as being unpatentable over Jun et al (US 
2001/0020981 Al) in view of Lin et al (US 6,307,964 Bl). 

Regarding claim 21, Jun discloses all elements as mentioned above in claim 1. Jun does 
not disclose using a Voronoi algorithm to determine the shape of the support to be placed on the 
canvas. 

Lin, in the same field of endeavor, teaches using a Voronoi algorithm to determine the 
shape of the support to be placed on the canvas (see col. 2, lines 41-46, representing a shape of 
an object in an image) 

It would have been obvious at the time the invention was made to one of ordinary skill in 
the art to modify the Jun reference to utilize a Voronoi algorithm as taught by Lin, in order 
increase the "effectiveness of a shape descriptor" by being "able to capture the structural 
information of the shape, yet be robust to noise within the boundary [to] yield the most favorable 
result" (see col. 2 lines 32-40). 
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Response to Arguments 
13. Applicant's arguments filed 1 1/8/07, in regards to claim 1 have been fully considered but 
they are not persuasive. Applicant argues that the Jun reference does not disclose the "filling in 
the space of the canvas" limitation of claim 1. This argument is not considered persuasive since 
it can be clearly seen in figure 13a and paragraphs 0074-0075 that the synthetic key frame FSK 
presents an image by combining key frames or key regions representing the entire content of the 
scene. The claim limitation, "filling in the space of the canvas", does not bring in the limitation 
of occupying every bit of space of the canvas. "Filling in the space of the canvas", only brings in 
the limitation of placing germs on a canvas which the Jun reference discloses. 

In regards to claim 2, applicant argues that the Jun with Uchihashi combination does not 
teach the amended claim limitation. This argument is not considered persuasive and the 
rejection of amended claim can be seen above. 

In regards to claim 10, applicant argues that the Jun reference does not meet the 
limitation, "filling in the space of the canvas with one or more parts of the image from the 
support". This argument is not considered persuasive since it can be clearly seen in figure 13a 
and paragraphs 0074-0075 that the synthetic key frame FSK presents an image by combining key 
frames or key regions representing the entire content of the scene. The claim limitation "filling 
in the space of the canvas with one or more parts of the image from the support", does not bring 
in the limitation of occupying every bit of space of the canvas. Furthermore, parts of the image 
from the support can be anything from a single extra pixel to a whole background image which 
in also seen in fig. 13a and paragraphs 0074-0075. "filling in the space of the canvas with one or 
more parts of the image from the support", only brings in the limitation of placing germs on a 
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canvas which the Jun reference discloses. Furthermore, the applicant argues that the Examiners 
admitted that in Jun there was no space or that it was filled with the germ, which is not accurate 
since in the interview summary and through further examination, the statement was not agreed 
upon and also stand untrue through further examination. 

In regards to claim 12, applicant argues that the claim is patentable since it depends from 
claim 10. This argument is not persuasive since the rejection of claim 10 stands and can be seen 
above. 

In regards to claims 7-9, 16-18, applicant argues that the claim limitations are not 
disclosed by the Jun with Li combination. This argument is not persuasive since the Jun with Li 
combination discloses the limitations and can be seen above in the rejection of the claims. 

In regards to claim 11, applicant argues that the claim is patentable since it depends from 
claim 10. This argument is not persuasive since the rejection of claim 10 stands and can be seen 
above. 

Conclusion 

14. Applicant's amendment necessitated the new ground(s) of rejection presented in this 
Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). 
Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within TWO 
MONTHS of the mailing date of this final action and the advisory action is not mailed until after 
the end of the THREE -MONTH shortened statutory period, then the shortened statutory period 
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will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 
CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, 
however, will the statutory period for reply expire later than SIX MONTHS from the date of this 
final action. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Edward Park whose telephone number is (571) 270-1576. The 
examiner can normally be reached on M-F 10:30 - 20:00, (EST). 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Vikkram Bali can be reached on (571) 272-7415. The fax phone number for the 
organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would 
like assistance from a USPTO Customer Service Representative or access to the automated 
information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 



/Edward Park/ 
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